Optimized avatar asset resource

ABSTRACT

Various customization options are provided for customizing a 3D avatar of a head. Features of the head and assets corresponding to the features can be customized using blend shapes. An amount of storage for the plurality of blend shapes is minimized by determining overlapping blend shapes that can be reused for a plurality of different assets. Further, techniques are provided for dynamic changes to an avatar in accordance with selected features and assets.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application claims the benefit of and priority to U.S.Provisional Application No. 62/679,875, filed Jun. 3, 2018, the entirecontents of which is incorporated herein by reference for all purposes.

FIELD

The present disclosure relates generally to reducing the amount ofresources required to create an avatar on a mobile device. Inparticular, the disclosure relates to creating various options forcreating an avatar while minimizing an amount of memory and runtimeresources that are used on the mobile device.

BACKGROUND

A user of a mobile device may desire to create an avatar. An avatar canbe a representation of a person, animal, or object. An avatar can becustomized as desired by users. User's would like to have a variety ofoptions for customizing an avatar. However, as the number of options forcustomizing an avatar increase, an amount of memory required to storethe and provide the options increases. It is a drain on the resources ofthe mobile device to continuously have all the customization optionsavailable. A mobile device can have limited memory and processingcapacity. Further, as the number of customization options increase, anamount of time required to load, change and display the avatar optionsalso increases. This can result in delays in modifying and updating theavatar. Delays in loading and providing the customization options canalso detract from the user experience.

Therefore, it is desired to provide a user with various avatarcustomization options while consuming minimal resources of a mobiledevice, such as memory space and processing time. Further, it is desiredto quickly provide the user with various options for customizing anavatar without slowing down the mobile device and thus providing anenhanced user experience.

BRIEF SUMMARY

Example embodiments provide techniques for optimizing avatar assetresources. Specifically, in example embodiments, an avatar can becreated which minimizes an amount of storage and processing required.

An example embodiment provides a method for optimizing asset resources.In accordance with an example embodiment, a mobile device can beconfigured to store a plurality of assets and a plurality of blendshapes corresponding with the plurality of assets for a plurality offeatures of an avatar. A plurality of features most frequently used by auser can be stored. The features that are most frequently used can bebased on a history of user interaction with the avatar application. Auser can input, on a user interface of the mobile device, a desiredfeature. The mobile device can receive a first feature selected by theuser from among a plurality of features that can be customized by theuser. The mobile device can identify a first set of the plurality ofassets that correspond to the first feature. The mobile device canreceive a selection of a first asset among the first set of theplurality of assets corresponding to the first feature and can identifya first set of the plurality of blend shapes that correspond to thefirst asset that is selected among the first set of the plurality ofassets corresponding to the first feature.

The mobile device can also receive a selection of a second asset amongthe first set of the plurality of assets corresponding to the firstfeature and can identify a second set of the plurality of blend shapescorresponding to the second asset that is selected among the first setof the plurality of assets corresponding to the first feature. The oneor more blend shapes of the first set of the plurality of blend shapesfor the first asset can overlap with one or more blend shapes of thesecond set of the plurality of blend shapes for the second asset.Therefore, a same blend shape can be reused for a plurality of assets.

In accordance with another example embodiment, a user can input adesired hairstyle for an avatar. The user interface can receive aselection of a first hairstyle for an avatar of a head. A first set ofzones on the avatar of the head, on which the first hairstyle ispositioned, can be determined. The first set of zones can be selectedfrom a plurality of possible zones for which hairstyles may be definedfor the avatar. A first set of blend shapes for each of the first set ofzones for the first hairstyle can be identified. The first set of blendshapes can relate to different directions of movement. First movementinformation for the head of the avatar can be received and a firstmovement of the first hairstyle can be stimulated using the firstmovement information. The simulation can be performed by a simulationengine that determines a linear combination of each first set of blendshapes based on the first movement information.

The mobile device can also receive a second hairstyle that the userwould like to apply to the avatar. The user interface of the mobiledevice can receive a selection of a second hairstyle for the avatar ofthe head. A second set of zones on the avatar of the head on which thesecond hairstyle is positioned can be determined. The second set ofzones can be selected from the plurality of possible zones for whichhairstyles may be defined for the avatar. The second set of zones caninclude a first portion that are in the first set of zones and a secondportion that are not in the first set of zones. A second set of blendshapes for the second hairstyle can be identified. The second set ofblend shapes can correspond to the second set of zones and relate to thedifferent directions of movement. Second movement information for thehead of the avatar can be received and the simulation engine cansimulate a second movement of the second hairstyle using the secondmovement information.

Other embodiments are directed to systems, portable consumer devices,and computer readable media associated with the methods describedherein.

A better understanding of the nature and advantages of exemplaryembodiments may be gained with reference to the following detaileddescription and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be readily understood by the following detaileddescription in conjunction with the accompanying drawings, wherein likereference numerals designate like elements, and in which:

FIGS. 1A-1H illustrate example user interfaces for creating andcustomizing an avatar, in accordance with some example embodiments.

FIGS. 2A, 2B, 2C and 2D illustrate a mesh on a 3D avatar of a head, inaccordance with some example embodiments.

FIG. 3 illustrates a configuration file for assets, in accordance withsome example embodiments.

FIG. 4 is an overview of a method for optimizing the storage of assetresources, in accordance with some example embodiments.

FIG. 5 is an overview of a method for optimizing asset resources, inaccordance with some example embodiments.

FIG. 6 illustrates a configuration file for hairstyles, in accordancewith some example embodiments.

FIG. 7 illustrates a block diagram of an example device, in accordancewith some embodiments.

FIG. 8 illustrates a simplified diagram of assets and overlapping blendshapes, in accordance with some example embodiments.

FIG. 9 illustrates an overview of blend shape storage and retrieval, inaccordance with some example embodiments.

FIG. 10 illustrates a method for simulating hair movement, in accordancewith some example embodiments.

FIGS. 11A-11D illustrate overlapping blend shapes, in accordance withsome example embodiments.

FIGS. 12A-12E illustrate corrective values, in accordance with someexample embodiments.

FIGS. 13A and 13B illustrate model swapping, in accordance with someexample embodiments.

FIGS. 14A-14E illustrate movement of hair within zones, in accordancewith some example embodiments.

FIG. 15 illustrates a flowchart for a method of generating a compositetexture, in accordance with some example embodiments.

FIG. 16 illustrates a method for efficient caching of geometry, inaccordance with some example embodiments.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofthe exemplary embodiments. However, it will be apparent that variousembodiments may be practiced without these specific details. Forexample, circuits, systems, algorithms, structures, techniques,networks, processes, and other components may be shown as components inblock diagram form in order not to obscure the embodiments inunnecessary detail. The figures and description are not intended to berestrictive.

I. User Interface

A user may desire to create an avatar. The avatar can be athree-dimensional (3D) representation of a person, such as the user, ananimal or an object. An avatar can be a partial representation (e.g., arepresentation of a head) and an avatar can be a complete representation(e.g, a whole body).

In the examples described, the avatar is of a three-dimensional head.However, aspects of the embodiments can be applied to avatars which arenot limited to a head. A 3D avatar can provide a more realisticrepresentation of a person or animal. Throughout the specification, the3D avatar may also be referred to as the head of the avatar or the 3Dhead. The 3D avatar can also be known as a 3D emoji or 3D animatedemoji. Motion capture from an imaging device can be used to create theanimation.

The 3D avatar can be created through an avatar application. The avatarapplication can downloaded by a user to their mobile device or theavatar application and be installed in the mobile device at the timewhen the mobile device is manufacture. The application can be dedicatedto the generation of an avatar or the application may include optionsfor generating an avatar (e.g., gaming application). The application canbe downloaded and stored on a mobile device by the user or theapplication can preinstalled on the mobile device. The mobile device canbe, for example, a mobile phone, smartwatch or a tablet.

When the application is selected, the user can be provided with a userinterface for generating an avatar. FIGS. 1A-1H illustrate example userinterfaces for creating and customizing an avatar, in accordance withsome example embodiments.

FIG. 1A illustrates a user interface 110 displaying options of avatarsthat can be customized by user. As shown in FIG. 1A, an avatar can beused to represent a human, animal, and/or objects. An avatar can be arealistic representation or can be a cartoonlike representation, such asan emoji. As shown in FIG. 1A, avatars can be directed to a human, suchas a man or a woman, a monkey, a robot, a rabbit, an alien, the unicorn,etc. These are merely examples and avatars can be directed to variousother animals, objects or things.

As shown in FIG. 1B, a user can customize various aspects of an avatarshead. For example, a user can customize features. Features can refer toa general category of items that can be customized. Features on anavatar's head can include face shape, hair, eyes, nose, ears, mouth, andchin of an avatar. Each of the features can include subcategories ofoptions that can be customized. For example, there can be differentfacial expressions, mouth expressions, eye expressions, accessories,hairstyles, etc. The subcategories for each of the features can bereferred to as assets. In some instances, an asset can also beconsidered a feature and a feature can also be considered an asset.

FIG. 1B illustrates a user interface 120 displaying ahead of an avatarbased on a selected face shape. As shown in FIGS. 1C and 1D, hairtexture, style and color can be customized. FIG. 1C illustrates a userinterface 130 including an avatar with a lighter hair color. FIG. 1Dillustrates a user interface 140 including an avatar with a darker haircolor. The selection of the assets can be controlled by using, forexample, a sliding bar. As the user slides the bar from left to rightthe hair color can change. Although a sliding bar is disclosed,selection of the assets can be controlled and selected other thanthrough a sliding bar.

Another feature for a head is texture. FIGS. 1C and 1D illustratesdifferent hair textures (e.g., flat, wavy, curly). A texture can includea flat image that is placed on the head of the avatar to indicatetexture. The flat image can be a decal or a sticker.

FIG. 1E illustrates a user interface 150 including a mouth that be canbe customized on the avatar. As shown in FIG. FIG. 1E, the “lips smilingmouth shape 151 and the “lips closing mouth shape” 152 can be an assetfor the feature “mouth.” Several different mouth shapes and sizes can beprovided to a user for customizing an avatar.

FIG. 1F illustrates a user interface 160 including accessories, such asearrings, hats, and glasses lips that be can be customized on theavatar. Accessories can include jewelry, such as earrings and otherpiercings, hats, and eyewear. Accessories can include other items thatcan be added to a head of a user.

FIG. 1G illustrates a user interface 170 including a side view of thehead of the avatar. FIG. 1H illustrates a user interface 180 including aclose-up side view of an earring location of the head of the avatar. Asshown in FIGS. 1G and 1H, different camera views for the features can beprovided. Therefore, in addition to providing the user with variousfeatures that can be customized, each of the features can also beprovided in different camera views.

As shown in FIGS. 1A-1H various features can be customized and variousfeatures can be animated. For example, a mouth can be animated to laugh,smile, frown, etc. An eye can be animated to blink, wink, open andclose, etc. Further, the mouth animations can be applied to varioussizes and shapes of mouths and the eye animations can be applied tovarious eye shapes, sizes, and colors. All of the customizations optionsthat are available to a user of the avatar application are preconfiguredand designed. The customization options can be configured and designedby, for example, the user of the avatar application or by an applicationdeveloper.

A user may want to manipulate features of the avatar using a camerasensor. As the user's physical features and position changes, a mobiledevice can detect the changes and modify the displayed image of theavatar to reflect the changes to the user's physical features and/orposition. A mobile device can detect a change in a face is a field ofview of one or more cameras of the mobile device. The mobile device canchange an appearance of an avatar based on a detected change in theface. Blend shapes can be configured for features of the avatar in orderto customize the movement and animation of the avatar. Blend shapes canmake the appearance of facial movements appear smoothly. Changing theappearance of the avatar based on the detected change in the faceprovides the user with options for controlling modifications to avirtual avatar without requiring the display of user interface controlelements.

II. Blend Shapes

In order to animate features on the avatar, a rig of the head iscreated. A rig is a skeleton of the head which enables the head to move.In an example embodiment, a single universal rig representing the headcan be used. The universal rig can also be known as a universal physicsrig. A mesh can be placed on the head so that the surface or the skin ofthe head can be modified. A mesh can include a certain number of pointsthat are used for sculpting the avatar. The mesh values can also beknown as the geometry for assets and features of the avatar. Therefore,all of the customization options (e.g., features and correspondingassets) only need to be generated for a single rig representing thehead. The same rig is used and different blend shapes can be triggered.

FIGS. 2A and 2B illustrate a mesh on a 3D avatar of a head, inaccordance with some example embodiments. FIG. 2A illustrates a frontview 200 of a mesh 201 on a 3D avatar of a head and FIG. 2B illustratesa back view 210 of the mesh 201 on a 3D avatar of a head

The mesh can include an array of vertices. In order to create ananimation, the mesh is morphed in order to produce the appearance of ananimation, and the morphing of the mesh results in blend shapes.Morphing of the mesh can also be called changing, modifying or animatingthe mesh.

FIGS. 2C and 2D illustrate a mesh that is morphed or changed. As shownin FIG. 2C the mesh is in a default state. The mesh in the area aroundthe mouth is at a position 251. The default state can be a neutral stateof the face. As shown in FIG. 2D, the mesh is morphed to indicate asmile. As shown in FIG. 2D, the mesh is morphed in the area 261 aroundthe mouth of the avatar. A blend shape is used to indicate the change inthe position of the mesh around the mouth. For example, FIG. 2D can be ablend shape that includes an offset from the original model shown inFIG. 2C.

A face can have, for example, 51 facial muscles. Each of the muscles cancorrespond to a blend shape or animation variable. For example, a smileor a blink can correspond to a facial muscle. The blend shapes thatcorrespond to the muscles can be known as expression shapes. Eachanimation variable can be associated with a single blend shape, however,some animation variables can be associated a plurality of blend shapes,or multiple animation variables can correspond to a single blend shape.The number of blend shapes can vary depending on the facial variable oranimation variable. Multiple animation variables could be changed to getto a particular blend shape from the default position of an asset.

For example, for a blink animation, the blink animation can use a blendshape for the eyes being open, another blend shape for the eyes beingclosed, and one or more blend shapes for the eyes partially closed(e.g., an in-between point). There will be fewer shapes (blend shape anddefault shape) than there are positions of the asset that are desiredfor displaying. For instance, the default position may correspond to theeyes being 50% open and the blend shapes correspond to the eyes being 0%open (fully closed) and 100% open (fully open), but it is stilldesirable to display an avatar with the eyes open 30%.

To enable rendering of these other positions, a linear combination canbe taken of the three shapes (default shape and two blend shapes). Asimulation (animation) engine can receive input that the eyes are to beopened at 30% (e.g., from an animation file or input from motion captureof a user), and then determine what weights should be applied to each ofthe shapes to provide a good approximation of what the eyes should looklike at 30% open. At a value of 30%, it is possible that the weight fromthe fully-open blend shape is 0 (e.g., since it is in the otherdirection from the default state of 50%), but change in shape can beover many mesh values. Thus, including a portion of the fully-open blendshape can provide a better approximation to the 30% open state. In someembodiments, a weight for a particular shape can be between 0 and 1, anda normalization process can be imposed such that the final weights addto 1.

Each position in a blinking animation can be approximated with adifferent linear combination of the three shapes. Interpolation can beperformed to obtain the shapes in between each of the blend shapes.Being “in between” can involve more than one dimension (e.g., input ofeyes being opened between 0-100%), and thus the input about the avatarposition can be a multidimensional point with each dimension beingbetween 0 and 1 (or other range). The simulation engine can thendetermine weights for any number of blend shapes to render the avatarimage corresponding to the multidimensional input position. Accordingly,the final expression at each instant in time can be a blend of the blendshapes.

As another example, in order to create an animation of a mouth smiling,the array of vertices for the mesh around the mouth can be moved fromtheir original position at a neutral state, to a second position inwhich the corners of the mouth are partially raised and to a thirdposition in which the corners of the mouth are fully raised. The changefrom the original neutral position of the mesh to the second position inwhich the corners are partially raised can correspond to one blendshape. The change from the second position in which the corners arepartially raised to the third position in which the corners of the mouthare fully raised can correspond to another blend shape.

It would require a lot of memory to store all of the blend shapes forall of the options of assets for each feature. However, certain blendshapes of different assets of a same feature may be shared. For example,a narrow eye and a wide eye may share some blend shapes. As anotherexample, blend shapes for mouth shapes and lips maybe shared. There canbe an overlapping shape to the mouth expression shapes. As a lip shapechanges (e.g, as the user moves their mouth), the change to the shape ofthe lip could result in a change to the shape of the mouth (e.g., outershape of mouth). For example, the size of the lips on the mouth shapemay be wider.

FIGS. 11A-11D illustrate overlapping blend shapes, in accordance withsome example embodiments. As shown in FIG. 11A, there can be a defaultmouth shape 1110 for an avatar. The default shape can also be known as adefault base shape. The user may desire the mouth shape to be a smilingmouth shape 1120, as shown in FIG. 11B. The smiling mouth shape can beknown as an expression shape since it relays an expression. A smile caninclude a left smile or a right smile expression shape. As shown in FIG.11C, a variant 1130 can be added to the smile mouth shape 1120. Thevariant is a change to the default base shape. The lip variant canchange the outer shape of the mouth and would be an overlapping shape tothe mouth expression shape. In FIG. 11C, the variant 1130 can be avariant to make the mouth wider. The additive result 1140 of combiningthe smiling mouth shape with the variant is shown FIG. 11D.

Therefore, in accordance with some example embodiments, blend shapes canoverlap instead of creating separate blend shapes for each asset. Therecan be two smile shapes, such as a left smile shape and a right smileshape, and six lip shapes that can overlap on the left and right smileshape. Instead of having twelve (12) separate blend shapes, the six lipshapes can be an additive to the two smile shapes. The determination asto whether or not to reuse a blend shape can be based on the visualappearance or intended shape of the asset or feature. For example, if apreviously established blend shape does not affect the visual appearanceor intended shape of an asset or feature, then the blend shape can bereused.

Further, in accordance with some example embodiments, instead of storingseparate blend shapes for each asset of each feature, certain blendshapes can be identified for reuse, thereby reducing the amount ofmemory required. Further, if a user selects a new asset of the samefeatures, one or more of the existing blend shapes in working memory canremain there as they will be reused for animating the new asset.

FIG. 8 illustrates a simplified diagram 800 of assets and overlappingblend shapes, in accordance with some example embodiments. FIG. 8 showsblend shapes for a first asset and blend shapes for a second asset inrelation to a three-dimensional space. In this example, there are 9possible blend shapes for the two assets of a feature. The blend shapesoccur at different locations for two dimensions in configuration(position) space. For example, a first dimension can correspond to howmuch a mouth is open vertically, and the second dimension can correspondto how much a mouth is open horizontally.

As shown in FIG. 8, asset #1 includes a subset of the possible blendshapes that includes blend shapes 1-6. Asset #2 includes a subset of thepossible blend shapes that includes blend shapes 4-9. The blend shapescorresponding to a particular asset can be identified by the developer,and encoded into a simulation engine. Thus, if asset #1 is selected, animage of the avatar having asset #1 (e.g., a particular mouth) at anyinstant in time (e.g., based on position from file or from motioncapture) can be rendered using the blend shapes 1-6 and the defaultshape for asset #1. And, if asset #2 is selected (e.g., a differentmouth), an image of the avatar having asset #2 at any instant in timecan be rendered using the blend shapes 4-9 and the default shape forasset #2.

In the example shown in FIG. 8, blend shapes 4, 5 and 6 are common toasset #1 and asset #2. Therefore, instead of duplicating blend shapes,blend shapes can be shared between assets. In this example, the storagerequirements would be reduced by 25%. The specific blend shapes that areshared can be selected judiciously to balance memory and visualappearance of the animation. And, the number of blend shapes that may beshared between two assets will depend on how similar the defaultappearance of two assets are to each other.

In some embodiments, the change to the expression on the avatar can bebased on facial inputs received from the user of the avatar application.As the user's physical features and position changes, a mobile devicecan detect the changes and modify the displayed image of the avatar toreflect the changes to the user's physical features and/or position. Thechange to the expression can also be entered by the user and may not bebased on the user's physical features and position. For example, ifasset #1 corresponds to a “mouth open” and asset #2 corresponds to a“mouth closed,” blend shapes 1, 2 and 3 may be unique to asset #1, blendshapes 7, 8 and 9 may unique to asset #2, and blend shapes 4, 5, and 6are common to asset #1 and asset #2.

Further, when the model is sculpted, only the array vertices that havemoved and their offsets need to be stored for a blend shape. An offsetis a change in the value of the points in the vertices from the originalvalues of the points on the mesh to a new position for a blend shape.The changes to the mesh can be stored as a series of vertex positions.The vertices can be interpolated for a series of vertex positions. Forexample, vertices for the creating a smile movement can be interpolated.Only the changes to the vertices in the original model are saved and theentire model does not need to be saved each time.

A. Variants, in-Betweens, and Corrective Shapes

Blend shapes of the example embodiments can be compatible with variants,in-betweens and corrective shapes. Variants, in-betweens and correctiveshapes are all forms of shape data. Certain features for an avatar maybe a blend shape that includes a variant. A variant can be a shape thatcan change a default base shape and is not an expression shape. Examplesof variants can include eye upturned, eye downturned, etc. As anotherexample, for a small nose, there may be a blend shape called variantsmall nose. A set of blend shapes can be stored that represent thedifferent variants for each feature. A set of variants can be stored fordifferent nose, ear, and eye sizes.

Corrective shapes are also a form of blend shapes. There may be someassets where the user desires to make adjustments to a blend shape. Forexample, if two assets do not appear together correctly, a correctiveshape can be applied to the original blend shapes. A corrective shape isa form of a blend shape that is an additional offset to an originalblend shape. A corrective shape is a shape that may not work alone andmay be activated on top of other shapes to fix or tune a combination ofshapes that would otherwise clash. When the original blend shape isadjusted, a corrective shape is created. The corrective shape is anoffset to the original blend shape. The corrective shape is anadditional offset to the original mesh. For example, a corrective shapecan be applied to a final combination of shapes such as a jaw and mouth,or an eye blink and an eye squint.

For example, an avatar may be sculpted to smile and may also be sculptedto have a jaw open. That is, two muscles are being sculpted at the sametime. However, when both of them are applied to the avatar, they may notappear correctly. The blended values of a corrective shape can depend onthe combination of the shapes that are being corrected. A correctivevalue can be equal to a minimum value of the shapes that are beingcorrected. For example, if JawOpen is 0.6 and MouthSmile is 0.9 then theassociated corrective value will be 0.6, which is the minimum value ofthe shapes that are being corrected.

FIGS. 12A-12E illustrate corrective values, in accordance with someexample embodiments. There may be a blend shape for an eye blink 1210,as shown in FIG. 12A, and a blend shape 1220 for an eye squint, as shownin FIG. 12B. Each eye can have its own blend shapes. Therefore, anavatar with two eyes can have a blend shape for each eye. Thecombination of the eye blink and the eye squint blend shapes may createa broken result 1230, as shown in FIG. 12C. As shown in FIG. 12C, thecombination of the eye blink and the eye squint results in an area 1231where the blend shapes appear deformed. Therefore, the eye blink and theeye squint cannot be combined without modification.

As shown in FIG. 12D, a corrective shape 1230 can be used to fix thebroken result 1230. As shown in FIG. 12E, when the corrective shape 1240is added to the broken result 1230, a corrected deformation 1250 can begenerated, which is a result of applying the corrective shape 1240 tothe broken result 1230. The user may determine a corrective shape thatcan be applied to the broken result. A plurality of differentpredetermined corrective shape options may be available to the user.

A corrective shape may be applied to the blend shape of the smile and/orthe jaw. The corrective shape can be applied to a model by adjusting acontrol dial on the user interface to a value between 0-1. The value 1can be the full corrective shape and the value 0 can the least fullversion of a corrective shape. For example, a value of 1 can be acorrection to mouth so that a mouth appears fully open and a value of 0can be a correction to the mouth so that the mouth appears closed.

If a modification is made to a first asset, e.g., the smile, it can bedetermined that there is a second asset, e.g., the jaw opening, thatcorresponds with the smile. Since these features correspond with eachother, a corrective shape can be calculated for the second asset (e.g.,jaw opening). The corrective shape is an offset of the blend shape ofthe second asset (e.g., jaw opening). The second asset (e.g., jawopening) can be updated in accordance with the corrective shape and theupdated second asset can be displayed.

There are also blend shapes called in-between blend shapes, which areblend shapes that are in-between blend shapes. An in-between shape canbe an interpolation shape through which the vertex will go throughbefore reaching the end point of the related parent shape. An example ofan in-between blend shape can be an eye that is partially open.

B. Consistency in Features and Assets

In accordance with example embodiments, consistency is maintainedbetween features and assets on an avatar since live tracking of changesis performed. If a change is made to one feature or asset, then blendshapes for the other features on the avatar are updated in accordancewith the change so that the avatar appears balanced. For example, if thesize of the head is increased or decreased (e.g., a variant is appliedto a blend shape), then blend shapes for the assets that are affected bythe change to the size of the head can automatically be changed. A userdoes not need to separately adjust each feature and/or asset and rebuildthe head.

For example, if a change is made to a head shape, such as a change to ajaw line, then assets that are affected by the change to the head shape,such as hair, will be updated in order to accommodate the change to thehead shape. Further, any additional assets that are affected by theinitial change to the head shape, can also be updated. As anotherexample, if a change is make to an eye shape, such as having the eyeshape upturned or downturned, then the eyelashes on the eye can also beupdated according to the new eye shape. The eyelashes can be updated tofollow the shape of the new eyelid for the changed eye. It can bevisually determined as to which assets are affected by a change and theaffected assets can be updated accordingly.

In determining which features and/or assets needs to be adjusted, themobile device determines which features and/or assets are being applied.For example, if a size of a head is changed, all assets that areavailable to a user are not updated because of the change to the size inthe head. Instead, it is determined which assets are being applied tothe avatar, and only those assets that are being applied need to beupdated. Therefore, not all of the assets for a feature need to beupdated, but only the assets for the feature that are currently used onthe avatar need to be updated.

The consistency in updating also applies to an animation. When there isan animation to the face, all assets that are affected by the animationwill also be updated without requiring the user to change all of theassets that are affected by the animation.

Therefore, an example embodiment provides a technique for enhancing theuser experience.

C. Configuration File—Blend Shapes

FIG. 3 illustrates a configuration file 300 for assets, in accordancewith some example embodiments. The configuration file 300 can be storedin a memory of the mobile device. A configuration file can indicate howfeatures, assets and blend shapes are connected with each other in orderto minimize the amount of memory consumed. A configuration file can bean XML file.

A mobile device can have limited memory and/or processing capacity.Therefore, it would consume a large amount of memory if all options arestored in the memory of the mobile device. In accordance with an exampleembodiment, a compressed file can be stored for each of assets. Thecompressed file can store the 3D geometry for the asset and thecompressed blend shapes. The files are compressed since the data isstored at a low resolution.

In storing the blend shapes, the deltas representing the changes to theinitial blend shape are stored, instead of storing all of the parametersof the updated blend shape. Only the vertices that have moved from thedefault model (delta values), are stored. The delta values, whichrepresent the changes from the original blend shape, can be quantized toa value between 0-1. That is, the deltas can be associated with adiscrete value.

As shown in FIG. 3, the file for the feature “eyes,” includes aplurality of configuration files for different assets. The configurationfile can include geometry shape information or blend shape informationfor an asset. An asset, such as mouth left up, would include blend shapeinformation. A configuration file for an asset, such as hairstyle andglasses, can include geometry information.

FIG. 3 shows configuration files for the asset eyes “blink shape” 310and “eyes wink shape” 320. Another example could be of a wide eye or asmall eye. In the example, the eyes in the configuration file are thesame and are directed to different expressions. The configuration filesfor the asset “eyes blink” 310 includes the blend shapes 311 in order tocreate the asset. The configuration file for the asset “eyes wink” 320includes the blend shapes 321 in order to create the asset. As shown inFIG. 3, blend shapes can be reused for different assets and they do notneed to be recreated for different assets.

Therefore, in accordance with example embodiments, space is optimizedand resources are shared across assets. A complete rebuild of the avatarfor each asset option is not required. Further, the amount ofinformation that is stored is also made minimal through the use of aunified rig. Providing multiple rigs can be very difficult to store andload. Therefore, the amount of data that is stored and loaded at runtimeis minimized through the use of a unified rig.

D. Method Overview—Blend Shapes

In accordance with some example embodiments, the amount of memory usedfor an avatar can be decreased by optimizing asset resources. FIG. 4 isan overview of a method 400 for optimizing the storage of assetresources, in accordance with some example embodiments.

At step 410, the mobile device identifies a first asset among the firstset of the plurality of assets corresponding to a first feature. Themobile device can perform the identification based on user input, e.g.,selecting a particular asset. The plurality of assets can be defined fora universal rig of a head. All of the features and assets will bedesigned in accordance with the single universal rig. When the assetsare defined for a single universal rig of the head, this can minimizethe amount of data that needs to be stored for each of the customizationoptions. Further, only one rig would need to be loaded to generate theavatar.

At step 420, the mobile device obtains the 3D geometry for the firstasset among the first set of the plurality of assets corresponding tothe first feature. The 3D geometry can include the mesh informationneeded to create the asset in a 3D space. The 3D geometry can include x,y, and z coordinate information.

At step 430, the mobile device compresses the 3D geometry for the firstasset. The 3D geometry can be compressed by filtering the 3D geometry toa lower resolution. The assets can be in a compressed state so that theywill consume less memory space on the mobile device.

At step 440, the mobile device identifies a first set of the pluralityof blend shapes corresponding to the first asset that is selected amongthe first set of the plurality of assets corresponding to the firstfeature. Blend shapes can be created for all of the features and assetsthat are available on the avatar application. As an example, an animatorcan sculpt blend shapes for the motion of a particular asset. Aplurality of positions can be selected for sculpting. The number ofpositions that are sculpted can vary based on an optimization procedure.The optimization procedure can be used to determine how close a positioncan be approximated by interpolation for different selections of theblend shapes. The optimization procedure can also be used to determinewhich blend shapes can be shared. Therefore, the optimization procedurecan provide sufficient accuracy for the motion of an asset.

At step 450, the mobile device stores the offset values for the firstset of the plurality of blend shapes corresponding to the first asset ina memory of the mobile device. An offset is a change in the value of thepoints in the vertices from the original values of the points on themesh. The changes to the mesh can be stored as a series of vertexpositions. Only the changes to the vertices in the original model aresaved and the entire model does not need to be saved each time. Acompressed configuration file is then created for each asset.

After configuration files are created for each asset, the assets can beused in creating an avatar. FIG. 5 is an overview of a method 500 foroptimizing asset resources, in accordance with some example embodiments.

At step 510, the mobile device obtains the store a plurality of assetsand the plurality of blend shapes corresponding with the plurality ofassets for a plurality of features of an avatar. The plurality of assetsand the plurality of blend shapes were stored as described in FIG. 4. Afeature for an avatar can correspond to, for example, an eye, a nose, anear, a mouth. If the feature is an eye, assets corresponding to thefeature (e.g., eye) can include eye size, eye shape, etc. Some assetsmay only require a single blend shape. For example, an open eye may onlyrequire a single blend shape for an open eye. However, a blinking eyemay require a plurality of blend shapes in order to achieve the visualof an eye blinding. Therefore, the application can be configured tostore a plurality of different assets and blend shapes for the assets.

At step 520, a user can select a feature that the user would like tocustomize. The user can enter a feature they would like to customize onthe user interface of the mobile device. For example, as shown in FIG.1E, the user can select to customize a feature such as a mouth.

At step 530, a processor on the mobile device can identify assets thatcorrespond to the feature that was selected by the user. For example,the mobile device can identify that 20 different mouth options areavailable. The identified assets can then be displayed to the user,e.g., as in FIG. 1E.

At step 540, the user can select a first asset that corresponds to thefeature. For example, as shown in FIG. 1E, the user can select an assetfor the mouth such as lips. As shown in FIG. 1E, the “mouth smilingmouth shape” 151 is an asset for the feature “mouth.”

At step 550, the application can identify blend shapes that correspondto the first asset. For example, if the first asset is a mouth smiling,there can be three blend shapes that are identified as corresponding tothe first asset of a mouth smiling. This is merely an example, and morethan three blend shapes can correspond to an asset. The blend shapescorrespond to different movement positions for how the mouth can beanimated. For example, one blend shape can be for the mouth beingopened, another blend shape for being closed (potentially tightly,depending on how the default 3D geometry is defined). Other blend shapescan correspond to various other movements, e.g., for other types offeatures, e.g., eyebrows being furled for an asset of eyes.

Once the blend shapes are identified, they can be used for animating theavatar. For example, if the user open her mouth, the animated characterof the avatar can open her mouth. An imaging sensors can use passive oractive techniques for such imaging. Passive techniques use ambientlight, as a typical camera would do. Changes in the camera images can beanalyzed to determine changes to the mouth. Active techniques can emitlight that is reflected from the face of the user, and then detected bysensors. For example, an array of infrared emitters can emit a beam oflight that reflects and is detected by an array of detectors. A time offlight can be measured for the light, thereby providing a depth map ofthe face. An opening of mouth can be detected by determining the area ofthe mouth, which does not reflect light is larger than in previousmeasurements.

The degree that the opening of the mouth has changed can be translatedto a numerical value. For example, an opening of 5 mm can translated tobe 20% of an open mouth. The precise correspondence between measurementsand percentage can vary per user's face, e.g., based on a width of theuser's mouth. Thus, for another user, an opening of 3 mm can correspondto the mouth being 20% open. Similar measurements can be made for awidth of a mouth, e.g., for a user pursing their lips.

The percentage that a mouth is open (or otherwise changed, e.g.,laterally for pursuing) can be used to determine which blend shapes touse to render the avatar to have a position that is similar to the ofthe user. The final rendered shape of the first asset can be representedas a linear combination of the blend shapes. To determine the linearcombination, a weight can be applied to each variable coordinate of ablend shape (e.g., 10 points for the mouth). Such a weight may be zero,but could be some fraction, e.g., 0.2. The default 3D geometry of theasset can also have an assigned weight. A second asset can be animatedusing a blend shape that is shared with or common with a first asset.The common blend shape can be kept in memory when the user selects a newasset.

At step 560, the user can select additional or alternative features tocustomize on the avatar. The user can select a second asset thatcorresponds to the feature. For example, for the feature “mouth” a usercan select an asset for the mouth such as lips. The “lip closing mouthshape” 152 that is an asset for the feature “mouth” can be selected.

At step 570, the application can identify blend shapes that correspondto the second asset. For example, if the second asset is a mouthclosing, there can be three blend shapes that are identified ascorresponding to the second asset of a mouth closing. This is merely anexample, and more than three blend shapes can correspond to an asset.

It can be determined that a the first asset and the second asset includea same blend shape. For example, one of the blend shapes for mouthsmiling can be the same as a blend shape for a mouth closing. Therefore,in accordance with some example embodiments, blend shapes that are thesame for different assets are reused. Similar blend shapes do not needto be recreated and can be shared between assets.

FIG. 9 illustrates an overview of blend shape storage and retrieval, inaccordance with some example embodiments. As shown in FIG. 9, blendshapes for various types of assets can be stored in persistent memory901. In the example shown in FIG. 9, 9 blend shapes are stored in thepersistent memory 901 for two different assets, and corresponds to theexample in FIG. 8. However, this is merely an example and blend shapesfor a plurality of different assets can be stored in memory.

At step 910 the first asset can be selected by a user via a userinterface, e.g., as described herein. A first asset can include blendshapes 1-6. Blend shapes that are associated with the first asset can beselected using blend shapes selection module 902.

At step 920, blend shapes that correspond to the first asset will beretrieved from persistent memory 901. Blend shapes selection module 902can include a list of identifiers for the blend shapes corresponding tothe first asset. The list can be specified by the provider of theanimation software and installed as part of a software package.

At step 930, blend shapes 1-6, which have been identified as the subsetof assets that correspond with the first asset, are stored in workingmemory 903. Once the blend shapes are stored in working memory 903, theymay be used for animation.

At step 940, the subset of blend shapes that correspond with the firstasset (e.g., blend shapes 1-6), can be provided to simulation module904. Simulation module 904 can simulate the expression of an avatar thatthat has the first asset on an avatar. For example, simulation module904 can determine a linear combination of blend shapes 1-6 and thedefault shape based on an expression position, e.g., as measured from auser's face using a motion capture sensor and circuitry.

At step 950, a second asset may be selected. A second asset can includeblend shapes 4-9. Blend shapes that are associated with the second assetcan be selected on blend shapes selection module 902.

At step 960, blend shapes that correspond to the second asset will beretrieved from persistent memory 901. Therefore, only blend shapes thatare not already in the working memory may be retrieved from persistencememory. It can be determined that blend shapes 4-6 are already inworking memory 903, and thus do not need to be retrieved. Thus, step 960can determine which blend shapes are not already in working memory 903.

At step 970, blend shapes 7, 8, 9, which have been identified as thesubset of assets that correspond with the second asset and are notcurrently stored in working memory 903, are stored in working memory903. If the blend shapes corresponding to the first asset are no longerneeded (e.g., blend shapes 1, 2, and 3), the blend shapes that are nolonger needed can be removed from working memory 903.

At step 980, the subset of blend shapes that correspond with the secondasset (e.g., blend shapes 4, 5, 6, 7, 8, 9), can be provided tosimulation module 904. Simulation module 904 can simulate the expressionassociated with the second asset on an avatar.

In some instances, it may be difficult to create blend shapes in orderto achieve a desired animation. Therefore, instead of morphing the meshusing blend shapes, a new mesh can be used or model swapping can beperformed.

III. Model Swapping Using Blend Shapes

Some assets can be based on physics shapes. In an example embodiment,hairstyles can be based on physics shapes. Physics shapes can also beknown as model swapping. There may be certain assets where it would betoo difficult to modify the original mesh in order to perform blendshaping to achieve a desired asset. Therefore, model swapping or physicsshapes can be used. For example, if the user is changing a hairstyle,the original mesh may not have sufficient points to create a differenthairstyle. It may be too time-consuming and may consume too manyresources to perform blend shaping to change a hairstyle since the meshthat the user is sculpting may not have a sufficient number of points totransform the mesh or geometry into the desired hairstyle. For example,it would be difficult to configure a bob hairstyle using a mesh for amohawk hairstyle, since a bob hairstyle would require more points on themesh than a mohawk hairstyle. Whereas if a user were modifying a shapeof a chin from a rounded chin to a square chin, it is more likely thatthe number of points on the mesh for the rounded chin can be used tocreate a blend shape for a square chin.

Therefore, in accordance with some example embodiments, when the userchanges a hairstyle, model swapping can be performed. Instead ofmorphing the original mesh, the original mesh is swapped with adifferent mesh in accordance with the desired asset. Blend shapes canalso be applied to the new mesh. However, by using model swapping, fewerblend shapes would be required since the new mesh can be more easilyconfigured to create the desired asset.

A. Hairstyles

Model swapping can be used with respect to hairstyles and hair movement.In model swapping, a different model is used instead of sculpting themodel using blend shapes. Various hairstyle options and hair textureoptions can be provided. For example, a user may be provided with 85different hairstyles to choose from. If the user is changing ahairstyle, the original mesh may not have sufficient points to create adifferent hairstyle. It may be too time-consuming and may consume toomany resources to perform blend shaping to change a hairstyle since meshthat the user is sculpting may not have a sufficient number of points totransform the mesh or geometry into the desired hairstyle.

Therefore, in accordance with some example embodiments, when the userchanges a hairstyle model swapping can be performed. Instead of morphingthe original mesh, the original mesh is swapped with a different mesh inaccordance with the desired asset. Blend shapes can also be applied tothe new mesh. However, fewer blend shapes would be required since thenew mesh can be more easily configured to create the desired asset.

In order to perform model swapping, a universal physics rig can be usedfor the head of the avatar. In order to achieve the desired movement,for example, hair movement, it would be difficult if a new physics rigis loaded for each hairstyle. Therefore, a single universal physics rigis used for a plurality of different hairstyle options. One physicssimulation engine that is always loaded and running can drive any of thehairstyle options. If a hairstyle is changed while the avatar of thehead is moving, the physics of the head responds correctly andconsistently. The physics of the head does not need to change when thehairstyles are changed. A plurality of hairstyles can be configured inaccordance with the universal physics rig.

FIGS. 13A and 13B illustrate model swapping, in accordance with someexample embodiments. FIG. 13A illustrates a mesh 1300 for a shorthairstyle. FIG. 13B illustrates a mesh 1310 for a mohawk. Bothhairstyles are configured in accordance with the universal physics rig.

Although a universal physics rig is described, some assets may require aseparate physics rig. Therefore, a separate physics rig may be generatedfor such assets. For example, a skull cap, which sits higher on a headthan hair, may have a separate physics rig.

The universal physics rig includes kinematic zones or kinematic chainsthat act like dynamic particles. The universal physics rig can include aplurality of predetermined zones. Each of the dynamic particles can beassociated with zones on the avatar. A zone can also be called a region.In an example embodiment, there are 25 dynamic particles in the head,however, this is an example, and there can be more or fewer dynamicparticles in the head. If there are 25 dynamic particles, then there canbe 25 zones on the head. The zones can cover all of the locations wherethe hair might be and each hairstyle can occupy different or overlappingzones. Each of the zones can be simulated with physics. The dynamicregions are configured to work in unison so that the resulting shapewill look normal when the hair is activated. The dynamic particles canbe predetermined areas of motion.

FIGS. 14A-14E illustrate movement of hair within zones, in accordancewith some example embodiments. FIG. 14A illustrates an avatar 1410 witha hairstyle (e.g., long hair with fringe) that is in a neutral shape.FIG. 14B illustrates an avatar 1420 with movement of the hair on thelower right side region of the hairstyle. FIG. 14C illustrates an avatar1430 with movement of the fringe on the upper right side region. FIG.14D illustrates an avatar 1440 with movement of the hair on the lowerleft side region. FIG. 14E illustrates an avatar 1450 with movement ofthe hair in the upper left side region of the avatar.

The 25 zones on a head can include the back left, back right, back,front left, front right, left, lower back left, lower back right, lowerback, lower front left, lower front right, lower front, lower left,lower right, right, tongue, upper back left, upper back right, upperback, upper front left, upper front right, upper front, upper left,upper right and upper. The 25 zones are locations on the head of theavatar. Although 25 zones are used in an example embodiment, anembodiment can be more or less than 25 zones.

Each of the zones can be associated with six degrees of movement. Thesix degrees of movement can include right, left, up, down, forward andbackward. Therefore, each zone (e.g., 25 zones corresponding to 25dynamic particles on the head) can include six degrees of movement.Blend shapes can be sculpted that correspond with the right, left, up,down, forward and backward movement for each zone.

Although there are 25 dynamic particles on the head, not all of thedynamic particles may be affected when an asset is applied. Eachhairstyle can apply to a predetermined number of zones and the hair inthe zones can react in accordance with the dynamic particles. Forexample, if there is a mohawk hairstyle, a mohawk hairstyle may covertwo zones, such as an upper back zone and a lower back zone. Therefore,the mohawk hairstyle will react in two different zones. For the twodifferent zones, there is a 3D movement. The hair located in each of thezones can move in a 3D axis (x, y, z). Further, blend shapes forforward, backward, up, down, left, and right movement for the hair foreach of the related zones can be determined.

As another example, a bob hairstyle may apply to 6 zones. In the exampleof the bob hairstyle, blend shapes for an up, down, left, right,forward, and backward movement will be stored for each of the six zonesto which the hairstyle applies. Each hairstyle can have a uniquegeometry and each hairstyle have unique sculpted blend shapes for thezones that the hairstyle includes.

Weights can be applied to each of the zones associated with a hairstyleto indicate the amount of movement for the hair located in the zone. Thevalue of the weight can be based on movement information received from auser. A processor of the mobile device can iterate over the blend shapesof the zones that are used for the selected hairstyle, and determine theblend shapes to be used and their corresponding weights to be appliedfor the occupied zones of a particular hairstyle.

Therefore, an example embodiment provides an efficient method ofsimulating movement of hair. An example embodiment would not require aseparate physics system for each hairstyle.

Although model swapping is described with respect to assets such ashairstyles, model swapping can be used for other assets such as facialhair, earrings, eyewear, and headwear.

However, these are merely examples. If a number of points on a mesh areinsufficient to generate a desired asset, then model swapping may beused instead of generating blend shapes.

B. Configuration File—Physics Shapes

In order to reduce the amount of memory needed to store the assets andtheir variants for the different hairstyles, an example embodimentprovides compressed file storage.

FIG. 6 illustrates a configuration file 600 for hairstyles, inaccordance with some example embodiments. As shown in FIG. 6, eachhairstyle has a compressed 3D file. The 3D file requires all theinformation necessary for generating a particular hairstyle. The 3D fileincludes the geometry and all the blend shapes for each hairstyle.

FIG. 6 illustrates a configuration file 600 for assets, in accordancewith some example embodiments. The configuration file 600 can be storedin a memory of the mobile device. A configuration file can indicate howfeatures, assets and blend shapes are connected with each other in orderto minimize the amount of memory consumed. As shown in FIG. 6, the filefor the asset “hairstyle,” for the feature “hair,” includes a pluralityof configuration files 610 and 620 for different hairstyles. FIG. 6shows configuration files for the asset “hairstyles_bob” 610 and“hairstyle_mohawk” 620. However, this is merely an example an there canbe configuration files for all the hairstyle options that are available.For example, there may be 85 configuration files if there are 85different hairstyle options available.

The configuration file 610 for the asset “hairstyles_bob” can includethe 3D geometry 611 and the zones 612 that are used by the asset“hairstyles_bob.” The configuration file 620 for the asset“hairstyles_mohawk” can include the 3D geometry 621 and the zones 622that are used by the asset “hairstyle_mohawk.”

In an example embodiment, a simulation can be performed on individualrigid bodies distributed around the head in locations related to thezones. The resulting dynamic that the individual rigid bodies receive isthen transferred to their respective zones. A rigid body drives eachzone. The resulting dynamic that is transferred to the zones can includea multiplication factor. The multiplication factor can vary based on thedesired dynamic. A custom multiplier value is created for each zone foreach of the hairstyles.

For example, a custom multiplier value can be created for each of the 25zones on a head for each hairstyle. For a given hairstyle, aconfiguration file can include a list of custom multipliers. Themultipliers are a mapping between the zone and how much to trigger theblend shape. Each zone in a hairstyle can be configured with a differentmultiplier based on the desired amount of movement. That is, each of thezones back left, back right, back, front left, front right, left, lowerback left, lower back right, lower back, lower front left, lower frontright, lower front, lower left, lower right, right, tongue, upper backleft, upper back right, upper back, upper front left, upper front right,upper front, upper left, upper right and upper, can include a custommultiplier value for each hairstyle. The multiplier can include adefault value and can be adjusted based on the desired amount ofmovement. For example, a default multiplier for a zone may be a value of0.3 and if more movement is desired, the default multiplier for the zonecan be modified to 0.35. A simulation factor of 0.35 would result ismore movement than a simulation factor of 0.3. The dynamic can beperformed by the simulation engine.

The 3D file can include a number of zones for the hairstyle and dataregarding the degrees of movement (e.g., up, down, left, right, forward,and backward movement) for each of the zones for the hairstyle. Thecompressed 3D files can be stored in a memory structure on the mobiledevice of the user.

FIG. 16 illustrates a method 1600 for efficient caching of geometry, inaccordance with some example embodiments. An example embodiment providesa method of caching or pre-computing the geometry for hairstyles so thathairstyles can be quickly displayed. At step 1610, the geometry for thehairstyles can be stored in low resolution. The low resolution geometryis stored as a sparse low resolution polynomial. At step 1620,operations for performing the subdivision can be precomputed and cachedso that the operations can be loaded quickly.

At step 1630, shaders can be generated and cached. Shaders, which can becode that defines the look of a material, such as skin reacting tolight, can also be generated and stored using the subdivision algorithm.Shaders can be written in metal shading language. In accordance with anexample embodiment, instead of compiling all of the shaders when ahairstyle is loaded, the shaders can be precompiled so as to speed upthe processing time between selection of the hairstyle and when thehairstyle is displayed.

At step 1640, a hairstyle can be selected for display. At step 1650,during runtime, the high resolution geometry is generated. The lowresolution geometry is subdivided so as to obtain smooth high resolutiongeometry. At runtime, the geometry is subdivided so that the hairstyleswill be generated using high resolution geometry that is very smooth.The amount of memory consumed by assets can be minimized since ahairstyle does not need to be recomputed and re-subdivided from scratchevery time the hairstyle is selected, since some of the geometry,operations, and shaders for a hairstyle has been previously cached.

C. Method Overview—Physics Shapes

FIG. 10 illustrates a method 1000 for simulating hair movement, inaccordance with some example embodiments.

At step 1010, a selection of a first hairstyle for an avatar of a headcan be received. The selection can be received on a user interface ofthe mobile device. For example, a user can select a mohawk hairstyle.

At step 1020, the mobile device determines a first set of zones on theavatar of the head on which the first hairstyle is positioned. The firstset of zones is selected from a plurality of possible zones for whichhairstyles may be defined for the avatar. For example, the first set ofzones can be from among 25 possible zones. A mohawk hairstyle cancorrespond to, for example, two zones.

At step 1030, a first set of blend shapes for each of the first set ofzones can be identified for the first hairstyle. The first set of blendshapes correspond to the first set of zones and relate to differentdirections of movement (e.g., up, down, forward, backward, left andright).

The direction of movement can correspond to a movement such as an upmovement, a down movement, a left movement, a right movement, a forwardmovement, and a backward movement.

At step 1040, first movement information for the head of the avatar isreceived. The first movement information can be received based on afacial recognition input from a user. Changes in the user's physicalfeatures and position changes can be received. Alternatively, a user caninput the first movement information manually.

At step 1050, a first movement of the first hairstyle can be simulatedusing the first movement information. The simulation can use asimulation engine that determines a linear combination of the first setof blend shapes based on the first movement information. The simulationof the first movement information can include, for each of the first setof zones and for each of the first set of blend shapes for the zone,determining a corresponding weight for the blend shape using the firstmovement information, and applying the corresponding weight to the blendshape as part of determining a linear combination of blend shapes forthe zone.

At step 1060, a selection of a second hairstyle for the avatar of thehead can be received. The selection can be received from the userinterface on the mobile device.

At step 1070, a second set of zones on the avatar of the head on whichthe second hairstyle is positioned is determined. The second set ofzones can be selected from the plurality of possible zones for whichhairstyles may be defined for the avatar. For example, the second set ofzones can be from among 25 possible zones. The second set of zones caninclude a first portion that are in the first set of zones and a secondportion that are not in the first set of zones. For example, a bobhaircut can correspond to, for example, six zones. Two of the six zonesof the bob hairstyle can overlap with the zones of the mohawk hairstyleand four of the six zones of the bob hairstyle do not overlap with thezones of the mohawk hairstyle.

At step 1080, a second set of blend shapes for the second hairstyle canbe identified. The second set of blend shapes can correspond to thesecond set of zones and can relate to the different directions ofmovement.

At step 1090, second movement information for the head of the avatar canbe received. The second movement information is different from the firstmovement information. The second movement information can be receivedbased on a facial recognition input from a user. Changes in the user'sphysical features and position changes can be received. Alternatively, auser can input the second movement information manually.

At step 1091, the simulation engine can simulate the second movement ofthe second hairstyle using the second movement information. Thesimulation can use a simulation engine that determines a linearcombination of the second set of blend shapes based on the secondmovement information. The simulation of the second movement of thesecond hairstyle can include, for each of the second set of zones andfor each of the second set of blend shapes for the zone, determining acorresponding weight for the blend shape using the second movementinformation, wherein the first weight of the first movement informationand the second weight of the second movement information are the same inresponse to the first movement information and the second movementinformation being the same. The corresponding weight to the blend shapecan be applied as part of determining a linear combination of blendshapes for the zone.

Swapping can be used in other areas of generating an avatar. Forexample, swapping can be used in the generation of textures for anavatar.

IV. Texture Swapping

In accordance with an example embodiment, the texture on the avatar ofthe head can be modified by texture swapping. Texture is an image thatis applied on the 3D model. The texture ask as decal or a sticker on the3D model. Textures can be painted on or layered onto the 3D model of thehead. Texture can be applied to, for example, skin or hair. For example,the face of an avatar can be modified so as to appear shiny orreflective. As another example, the face of an avatar can be modified towas to appear to have the growth of facial hair (e.g., 5 o'clockshadow). Texture swapping can also be performed with eyebrows,eyelashes, facial hair, etc.

It may be difficult to generate blend shapes for texture. Therefore, inaccordance with some example embodiments, changing the texture isperformed by texture swapping. Similar to model swapping, certaintextures may be difficult to achieve using blend shapes. Therefore,instead of generating blend shapes, a texture on the face can be swappedwith a different texture. A texture for a head is composed of layers.Some of the layers can be shared and/or constant. An example of a sharedand constant layer is the inner mouth color. Some textures are variantbased, such as eyebrows.

A texture can be composited for the combined assets on the avatar. Forexample, an avatar may have facial hair texture (e.g., 5 o'clockshadow), eyebrow texture, lipstick texture, hairstyle texture (e.g.,buzz haircut), etc. In accordance with an example embodiment, instead ofcompositing the texture for facial hair texture, eyebrows texture,lipstick color texture, and hairstyle texture separately, a singletexture can be applied for all of the assets. Therefore, texture can bequickly applied to the avatar.

FIG. 15 illustrates a flowchart for a method 1500 of generating acomposite texture, in accordance with some example embodiments. At step1510, all of the assets for which texture is to be applied, areidentified. Some or all assets on an avatar may be identified by a userfor applying texture. At step 1520, the texture for all of theidentified assets are composited into a single texture. That is, asingle texture composite is created that is to be applied to all of theidentified assets. At step 1530, the composited texture is stored in abuffer for temporary storage before the composited texture is applied tothe model. At step 1530, the composited texture is applied to the 3Dmodel.

The complexity of the textures can affect the amount of memory that isused. Therefore, the amount of memory that is used for textures can beminimized when a single composite texture is created and stored in abuffer as opposed to creating and storing a plurality of textures for aplurality of different assets of the model independently.

V. Runtime Optimization

In addition to requiring minimal memory storage, an example embodimentfor provides for a minimal use of resources of the mobile device duringruntime. Various optimization techniques can be employed while theapplication is running. Optimization techniques can include generatingand storing the images at a low resolution, interpolation, an on-screendisplay cache, and a thumbnail cache.

A. Low Resolution and Interpolation

Assets are stored at a low resolution and are smoothed (e.g.,interpolation) at runtime to be higher resolution. Low resolution can bemaintained by using a subdivision surface so that the mesh can be storedat a low resolution. Quizzing can also be used, which allows for thesharpening of edges with a very small amount of information.

Each of the assets are stored in small compressed file. Each asset filehas one piece of geometry. For example, each hairstyle has its own smallcompressed file with its piece of geometry. Further, each eye shape andmouth shape can have its own compressed file. During runtime, since asingle rig for the head is used, only a file corresponding to the singlehead needs to be loaded. The asset files that are loaded and unloadedwill vary based on the assets that are selected by the user on the userinterface. If the user selects a certain feature, assets correspondingto the feature can be loaded into memory. Assets corresponding tofeatures that are not selected by user will not be loaded into memory.

Since the assets are split into different files, it can be easy tocombine different assets when creating an avatar. For example, aplurality of different complete avatars having the different assetsoptions do not need to be stored and/or loaded into memory. Therefore,the processing time and the amount of resources used by the mobiledevice can be decreased. Further, by having small compressed files,files for the assets can be easily loaded and unloaded from memory.Assets that are currently being applied to the avatar can be loaded tomemory and when the asset is no longer being applied to the avatar, theasset can be unloaded from memory.

B. On-Screen Display (OSD) Cache

Since there are various feature an asset combinations that can beselected, only those that will appear on the display at a given time canbe cached. For example, if there are 100 hairstyle options, all 100hairstyle options do not need to be cached in response to the userselecting feature “hair.” Instead, only those hairstyles that willappear on the display at a given time need to be cached.

In an example embodiment, only the hairstyles of an active instantiatedavatar are cached. That is, only the hairstyles of the avatars that aredisplayed live on the screen, are cached. In another embodiment, duringhairstyle selection, all of the hairstyles can be kept in memory so thatchanges to the hairstyle can be reflected quickly. For example, bykeeping all of the hairstyles in memory, changes to hair color can bereflected quickly. All of the thumbnails representing the hairstyles canbe quickly recomputed based on the change in hair color.

C. Thumbnail Cache

As shown in FIGS. 1B-1H, various thumbnails are displayed in accordancewith a current state of the avatar. As the user scrolls through thevarious asset obvious, it is beneficial to see the asset options inrelation to the current avatar. Therefore, an example embodimentprovides a thumbnail cache so the assets can be displayed quickly withrespect to the current avatar.

The content that is displayed on the user interface can be dynamicallybuilt based on the configuration file for the assets. Thumbnails arerendered dynamically as different assets are selected. As shown in FIG.1D, since the lighter hair color is selected for the avatar, all of thethumbnail images that are displayed with different assets, are displayedwith the lighter hair color that has been selected. As shown in FIG. 1D,when the darker hair color is selected for the avatar, all of thethumbnail images are displayed with different assets are displayed withthe darker hair color that has been selected.

Therefore, the content that is displayed on the user interface is basedon the choices that are made by the user. The thumbnails that appear onthe interface are dynamic based on user actions and are not static.

In order to enable the dynamic changes to the thumbnails, optimizationtechniques are employed. The avatar including the assets arepre-computed or cached. Computations for rendering the asset can be doneahead of time, such as subdivision data caching. The thumbnails can bepre-compiled and subdivision can be precomputed or partially precomputedin order to expedite the rendering of the thumbnails. Since assets havebeen pre-computed or cached, when the user selects a particular asset,less time is required to completely reference the asset. For example, ifthe user selects the feature “hair,” a plurality of different hairstyles(assets) are pre-computed on the current avatar the feature “hair.”

In an example embodiment, when a user selects a particular head shape,features and assets that can be selected can be displayed in thumbnails.As shown in FIGS. 1G and 1H, the thumbnails that are displayed caninclude customizations that have already been made by a user. Earringsthumbnails 171 are thumbnails of a current state of an avatar withdifferent earrings options. Hat thumbnails 172 are thumbnails of acurrent state of an avatar with different hat options. After the avatarof the user is loaded onto the monitor, the various customizationoptions that they user may apply to their avatar are pre-cached. Whenthe user selects a particular asset, the possible asset options arerendered quickly. Therefore, in accordance with some exampleembodiments, the user can easily see how other customizations wouldappear on their currently designed avatar.

VI. Camera and Assets

The images that are displayed in the thumbnails can be displayed indifferent 3D views. The different views can be called different cameraviews. The camera views are three-dimensional. All of the assets thatare displayed on the display, such as the thumbnails, can be displayedin accordance with the selected camera view.

FIG. 1G illustrates a user interface 170 displaying earring accessoriesin a profile camera view 173. FIG. 1H illustrates a user interface 180displaying earring accessories in a close-up profile camera view. Thecamera views shown in FIGS. 1G and 1H are merely examples and differentcamera views and different camera angles can be made available to auser. A user may be provided with various camera view options. Thenumber of views can vary based on user preferences. Example camera viewscan include a 3D camera view of the head from the front, a 3D cameraview of the head from the back, a 3D camera view of the eyes, a 3Dcamera view for a chin, a 3D camera view for a nose, 3D camera view forears, etc. All of the assets that can be selected by the user can bedisplayed in accordance with the selected camera view.

In accordance with example embodiments, the assets displayed in thedifferent camera views are updated in real time in response to changesmade by the user creating the avatar.

VII. Rule System

In some situations, assets may not be compatible or may clash. Forexample, a user may want to have a particular hairstyle, but they wouldalso like to add a hat. However, adding a hat to a hairstyle may affectthe appearance of the hairstyle. As another example, a user may want toplace sunglasses over an avatar's eyes or hair. However, adding thesunglasses can affect the appearance of the eyes or hair.

Therefore, an example embodiment provides a rule system in the event ofpossible incompatibilities between assets. Specifically, configurationrules are configured for each combination of assets to avoid any clashesor conflicts. A universal hairline is created for the head. Differentcombinations of hairstyles and hats are configured in accordance withthe universal hairline. All the hats can pinch at the same location onthe universal hairline. For example, a bob hairstyle can configured for50 different hats. Further, all of the hairstyles are configured so thatthey fall within the hairline. When a hat is placed on top of the hair,the hair that falls under the hat is compressed. Further, the hats thatis placed on top of the hair can be modified to as to provide space forthe hair to fall under. In addition, if a hat is placed so as to coverpart of the hair, then the portion of the hair that is covered by thehat is updated accordingly. Therefore, with respect to the hair and hatexample, there is a rule system for how different hairstyles will affectdifferent hats, and there is a rule system for how different hats willaffect different hairstyles.

Rules can refer to the dependencies between assets. A rule can includean identifier for an asset, dependencies, and variants. For example,when a hairstyle identified as “hair-straight_long_down_fringe” isselected, a preset ear can be overridden since the hairstyle will affectthe ears. Further, a variant can be activated for headwear based on thehairstyle.

The selection of the different hats on hairstyles are displayed onthumbnails in real-time so that a user can readily see the possibleoptions. For example, if the user has selected a particular hairstyle(e.g., bob), then all of the hats will be applied in accordance with therules for the particular hairstyle. If, for example, the user hasselected a particular hat (e.g., baseball cap) and is in the process ofselecting a hairstyle, the available hairstyle options will conform tothe rules of the hat. Further, the thumbnails that are displayed aredynamic in accordance with the changes made by the user. If a change ismade to the avatar, then thumbnail images that are displayed will beconsistent with the change made to the avatar.

VIII. Example Device

FIG. 7 is a block diagram of an example electronic device 700. Device700 generally includes computer-readable medium 702, a processing system704, an Input/Output (I/O) subsystem 706, wireless circuitry 708, andaudio circuitry 710 including speaker 712 and microphone 714.Computer-readable medium 702 can be a non-transitory computer-readablemedium. These components may be coupled by one or more communicationbuses or signal lines 703. Device 700 can be any portable electronicdevice, including a handheld computer, a tablet computer, a mobilephone, laptop computer, tablet device, media player, personal digitalassistant (PDA), a key fob, a car key, an access card, a multi-functiondevice, a mobile phone, a portable gaming device, or the like, includinga combination of two or more of these items.

It should be apparent that the architecture shown in FIG. 7 is only oneexample of an architecture for device 700, and that device 700 can havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIG. 7 can be implemented inhardware, software, or a combination of both hardware and software,including one or more signal processing and/or application specificintegrated circuits.

Wireless circuitry 708 is used to send and receive information over awireless link or network to one or more other devices' conventionalcircuitry such as an antenna system, an RF transceiver, one or moreamplifiers, a tuner, one or more oscillators, a digital signalprocessor, a CODEC chipset, memory, etc. In some embodiments, wirelesscircuitry 708 is capable of establishing and maintaining communicationswith other devices using one or more communication protocols, includingtime division multiple access (TDMA), code division multiple access(CDMA), global system for mobile communications (GSM), Enhanced Data GSMEnvironment (EDGE), wideband code division multiple access (W-CDMA),Long Term Evolution (LTE), LTE-Advanced, WiFi (such as IEEE 802.11a,IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), Bluetooth, Wi-MAX,voice over Internet Protocol (VoIP), near field communication protocol(NFC), a protocol for email, instant messaging, and/or a short messageservice (SMS), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument. A mobile device can include wireless circuitry that cancommunicate over several different types of wireless networks dependingon the range required for the communication. For example, a short-rangewireless transceiver (e.g., Bluetooth), a medium-range wirelesstransceiver (e.g., WiFi), and/or a long range wireless transceiver(e.g., GSM/GPRS, UMTS, CDMA2000 1×/EV-DO and LTE/LTE-Advanced) can beused depending on the type of communication or the range of thecommunication.

Wireless circuitry 708 is coupled to processing system 704 viaperipherals interface 716. Peripherals interface 716 can includeconventional components for establishing and maintaining communicationbetween peripherals and processing system 704. Voice and datainformation received by wireless circuitry 708 (e.g., in speechrecognition or voice command applications) is sent to one or moreprocessors 718 via peripherals interface 716. One or more processors 718are configurable to process various data formats.

Peripherals interface 716 couple the input and output peripherals ofdevice 700 to the one or more processors 718 and computer-readablemedium 702. One or more processors 718 communicate withcomputer-readable medium 702 via a controller 720. Computer-readablemedium 702 can be any device or medium that can store code and/or datafor use by one or more processors 718. Computer-readable medium 702 caninclude a memory hierarchy, including cache, main memory and secondarymemory. The memory hierarchy can be implemented using any combination ofRAM (e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and/or opticalstorage devices, such as disk drives, magnetic tape, CDs (compact disks)and DVDs (digital video discs). In some embodiments, peripheralsinterface 716, one or more processors 718, and controller 720 can beimplemented on a single chip, such as processing system 704. In someother embodiments, they can be implemented on separate chips.

Processor(s) 718 can include hardware and/or software elements thatperform one or more processing functions, such as mathematicaloperations, logical operations, data manipulation operations, datatransfer operations, controlling the reception of user input,controlling output of information to users, or the like. Processor(s)718 can be embodied as one or more hardware processors, microprocessors,microcontrollers, field programmable gate arrays (FPGAs),application-specified integrated circuits (ASICs), or the like.

Device 700 also includes a power system 742 for powering the varioushardware components. Power system 742 can include a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light emittingdiode (LED)) and any other components typically associated with thegeneration, management and distribution of power in mobile devices.

In some embodiments, device 700 includes a camera 744. In someembodiments, device 700 includes sensors 746. Sensors can includeaccelerometers, compass, gyrometer, pressure sensors, audio sensors,light sensors, barometers, and the like. Sensors 746 can be used tosense location aspects, such as auditory or light signatures of alocation. In some embodiments, device 700 can include a GPS receiver,sometimes referred to as a GPS unit 748. A mobile device can use asatellite navigation system, such as the Global Positioning System(GPS), to obtain position information, timing information, altitude, orother navigation information. In some embodiments, device 700 caninclude external port 736 (e.g., USB, FireWire, Lightning connector,110-pin connector, etc.). External port 736 can be adapted for couplingdirectly to other devices or indirectly over a network (e.g., theInternet, wireless LAN, etc.).

One or more processors 718 run various software components stored incomputer-readable medium 702 to perform various functions for device700. In some embodiments, the software components include operatingsystem 722, communication module (or set of instructions) 724, locationand motion module 726, other applications (computer products, sets ofinstructions, etc.) 728, and haptic feedback module 730 and blend shapeinput module 732. Operating system 722 can be any suitable operatingsystem, including iOS, Mac OS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS,or an embedded operating system such as VxWorks. The operating systemcan include various procedures, sets of instructions, softwarecomponents and/or drivers for controlling and managing general systemtasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 724 facilitates communication with other devicesover external port 736 or via wireless circuitry 708 and includesvarious software components for handling data received from wirelesscircuitry 708 and/or external port 736.

The one or more applications 734 on device 700 can include anyapplications installed on the device 700, including without limitation,a browser, address book, contact list, email, instant messaging, socialnetworking, word processing, keyboard emulation, widgets, JAVA-enabledapplications, encryption, digital rights management, voice recognition,voice replication, a music player (which plays back recorded musicstored in one or more files, such as MP3 or AAC files), etc.

The state information 728 can include a power state or mode of operationcollected about one or more components of device 700. State information728 can include a power state or mode of operation of components ofother devices obtained by device 700. As the mobile device or thecompanion device, device 700 can generate local state information andremote state information in state information 738. Device 700 (e.g.,using wireless circuitry 708) can use state information 728 to manageoperations locally in order to reduce power usage. Device 700 cansynchronize all or part of the local state information in stateinformation 728 with the other devices. The other devices can delaysending messages to device 700 allowing processors 718 to remain in asleep state further reducing power consumption.

There may be other modules or sets of instructions (not shown), such asa graphics module, a time module, etc. For example, the graphics modulecan include various conventional software components for rendering,animating and displaying graphical objects (including without limitationtext, web pages, icons, digital images, animations and the like) on adisplay surface. In another example, a timer module can be a softwaretimer. The timer module can also be implemented in hardware. The timemodule can maintain various timers for any number of events.

I/O subsystem 706 can be coupled to a display system (not shown), whichcan be a touch-sensitive display. The display displays visual output tothe user in a GUI. The visual output can include text, graphics, video,and any combination thereof. Some or all of the visual output cancorrespond to user-interface objects. A display can use LED (lightemitting diode), LCD (liquid crystal display) technology, or LPD (lightemitting polymer display) technology, although other displaytechnologies can be used in other embodiments.

In some embodiments, I/O subsystem 706 can include a display and userinput devices such as a keyboard, mouse, and/or trackpad. In someembodiments, I/O subsystem 706 can include a touch-sensitive display. Atouch-sensitive display can also accept input from the user based atleast part on haptic and/or tactile contact. In some embodiments, atouch-sensitive display forms a touch-sensitive surface that acceptsuser input. The touch-sensitive display/surface (along with anyassociated modules and/or sets of instructions in computer-readablemedium 702) detects contact (and any movement or release of the contact)on the touch-sensitive display and converts the detected contact intointeraction with user-interface objects, such as one or more soft keys,that are displayed on the touch screen when the contact occurs. In someembodiments, a point of contact between the touch-sensitive display andthe user corresponds to one or more digits of the user. The user canmake contact with the touch-sensitive display using any suitable objector appendage, such as a stylus, pen, finger, and so forth. Atouch-sensitive display surface can detect contact and any movement orrelease thereof using any suitable touch sensitivity technologies,including capacitive, resistive, infrared, and surface acoustic wavetechnologies, as well as other proximity sensor arrays or other elementsfor determining one or more points of contact with the touch-sensitivedisplay.

Further, I/O subsystem 706 can be coupled to one or more other physicalcontrol devices (not shown), such as pushbuttons, keys, switches, rockerbuttons, dials, slider switches, sticks, LEDs, etc., for controlling orperforming various functions, such as power control, speaker volumecontrol, ring tone loudness, keyboard input, scrolling, hold, menu,screen lock, clearing and ending communications and the like. In someembodiments, in addition to the touch screen, device 700 can include atouchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad can be a touch-sensitive surface that is separatefrom the touch-sensitive display or an extension of the touch-sensitivesurface formed by the touch-sensitive display.

The foregoing description may make reference to specific examples of amobile device (e.g., a wrist-worn device) and/or a companion device(e.g., a smart phone). It is to be understood that these examples areillustrative and not limiting; other devices can be substituted and canimplement similar functional blocks and/or algorithms to performoperations described herein and/or other operations.

Embodiments of the present invention, e.g., in methods, apparatus,computer-readable media and the like, can be realized using anycombination of dedicated components and/or programmable processorsand/or other programmable devices. The various processes describedherein can be implemented on the same processor or different processorsin any combination. As used herein, a processor includes a single-coreprocessor, multi-core processor on a same integrated chip, or multipleprocessing units on a single circuit board or networked. Wherecomponents are described as being configured to perform certainoperations, such configuration can be accomplished, e.g., by designingelectronic circuits to perform the operation, by programmingprogrammable electronic circuits (such as microprocessors) to performthe operation, or any combination thereof. Further, while theembodiments described above may make reference to specific hardware andsoftware components, those skilled in the art will appreciate thatdifferent combinations of hardware and/or software components may alsobe used and that particular operations described as being implemented inhardware might also be implemented in software or vice versa.

Computer programs incorporating various features of the presentinvention may be encoded on various computer readable storage media;suitable media include magnetic disk or tape, optical storage media suchas compact disk (CD) or DVD (digital versatile disk), flash memory, andthe like. Computer readable storage media encoded with the program codemay be packaged with a compatible device or provided separately fromother devices. In addition program code may be encoded and transmittedvia wired optical, and/or wireless networks conforming to a variety ofprotocols, including the Internet, thereby allowing distribution, e.g.,via Internet download. Any such computer readable medium may reside onor within a single computer product (e.g. a hard drive, a CD, or anentire computer system), and may be present on or within differentcomputer products within a system or network. A computer system mayinclude a monitor, printer, or other suitable display for providing anyof the results mentioned herein to a user.

Although specific example embodiments have been described, variousmodifications, alterations, alternative constructions, and equivalentsare also encompassed within the scope of the invention. Embodiments ofthe present invention are not restricted to operation within certainspecific data processing environments, but are free to operate within aplurality of data processing environments. Additionally, althoughembodiments of the present invention have been described using aparticular series of transactions and steps, it should be apparent tothose skilled in the art that the scope of the present invention is notlimited to the described series of transactions and steps. Variousfeatures and aspects of the above-described embodiments may be usedindividually or jointly.

As described above, one aspect of the present technology is thegathering and use of data available from various sources in order togenerate an avatar. The present disclosure contemplates that in someinstances, this gathered data may include personal information data thatuniquely identifies or can be used to contact or locate a specificperson. Such personal information data can include demographic data,location-based data, telephone numbers, email addresses, twitter ID's,home addresses, data or records relating to a user's health or level offitness (e.g., vital signs measurements, medication information,exercise information), date of birth, or any other identifying orpersonal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used toprovide customized avatar options that is of greater interest to theuser. Accordingly, use of such personal information data can enableusers to have more personally customized avatars. Further, other usesfor personal information data that benefit the user are alsocontemplated by the present disclosure. For instance, health and fitnessdata may be used to provide insights into a user's general wellness, ormay be used as positive feedback to individuals using technology topursue wellness goals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, the presenttechnology can be configured to allow users to select to “opt in” or“opt out” of participation in the collection of personal informationdata during registration for services or anytime thereafter. In additionto providing “opt in” and “opt out” options, the present disclosurecontemplates providing notifications relating to the access or use ofpersonal information. For instance, a user may be notified upondownloading an app that their personal information data will be accessedand then reminded again just before personal information data isaccessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, suggestedcustomizations can be selected and delivered to users by inferringpreferences based on non-personal information data or a bare minimumamount of personal information, such as the content being requested bythe device associated with a user, other non-personal informationavailable to the avatar application, or publicly available information.

Further, while example embodiments have been described using aparticular combination of hardware and software, it should be recognizedthat other combinations of hardware and software are also within thescope of the example embodiments. Some of the example embodiments may beimplemented only in hardware, or only in software, or using combinationsthereof. The various processes described herein can be implemented onthe same processor or different processors in any combination.Accordingly, where components or modules are described as beingconfigured to perform certain operations, such configuration can beaccomplished, e.g., by designing electronic circuits to perform theoperation, by programming programmable electronic circuits (such asmicroprocessors) to perform the operation, or any combination thereof.Processes can communicate using a variety of techniques including butnot limited to conventional techniques for interprocess communication,and different pairs of processes may use different techniques, or thesame pair of processes may use different techniques at different times.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that additions, subtractions, deletions, and other modificationsand changes may be made thereunto without departing from the broaderspirit and scope as set forth in the claims. Thus, although specificexample embodiments have been described, these are not intended to belimiting. Various modifications and equivalents are within the scope ofthe following claims.

All patents, patent applications, publications, and descriptionsmentioned herein are incorporated by reference in their entirety for allpurposes. None is admitted to be prior art.

What is claimed is:
 1. A method comprising: storing, by a mobile device,a plurality of assets and a plurality of blend shapes corresponding withthe plurality of assets for a plurality of features of an avatar,wherein the plurality of assets are defined for a same rig; receiving,on a user interface of the mobile device, a selection of a first featurefrom among the plurality of features; identifying a first set of theplurality of assets for the same rig that corresponds to the firstfeature; receiving, on the user interface of the mobile device, aselection of a first asset among the first set of the plurality ofassets corresponding to the first feature; identifying a first set ofthe plurality of blend shapes corresponding to the first asset of thefirst feature that is selected among the first set of the plurality ofassets corresponding to the first feature; receiving, on the userinterface of the mobile device, a selection of a second asset of thefirst feature among the first set of the plurality of assetscorresponding to the first feature; and identifying a second set of theplurality of blend shapes corresponding to the second asset of the firstfeature that is selected among the first set of the plurality of assetscorresponding to the first feature, wherein one or more blend shapes ofthe first set of the plurality of blend shapes for the first asset ofthe first feature overlaps with one or more blend shapes of the secondset of the plurality of blend shapes for the second asset of the firstfeature.
 2. The method according to claim 1, wherein the avatar is arepresentation of a head.
 3. The method according to claim 1, whereinthe storing the plurality of assets and the plurality of blend shapescorresponding with the plurality of assets for the plurality of featurescomprises: identifying the first asset among the first set of theplurality of assets corresponding to the first feature; obtaining 3Dgeometry for the first asset among the first set of the plurality ofassets corresponding to the first feature; compressing the 3D geometryfor the first asset; identifying the first set of the plurality of blendshapes corresponding to the first asset that is selected among the firstset of the plurality of assets corresponding to the first feature; andstoring offset values for the first set of the plurality of blend shapescorresponding to the first asset.
 4. The method according to claim 1,wherein a same blend shape is used for the one or more blend shapes ofthe first set of the plurality of blend shapes for the first asset thatoverlaps with one or more blend shapes of the second set of theplurality of blend shapes for the second asset.
 5. The method accordingto claim 1, further comprising: receiving a modification to the firstasset in the first set of the plurality of assets corresponding to thefirst feature; determining whether there is a second asset in a secondset of the plurality of assets of a second feature that corresponds withthe first asset in the first set of the plurality of assetscorresponding to the first feature; in response to determining thatthere is the second asset in the second set of the plurality of assetsof the second feature corresponding with the first asset in the firstset of the plurality of assets corresponding to the first feature,calculating a corrective shape for the second asset, wherein thecorrective shape is an offset of a blend shape of the second asset;updating the second asset in the second set of the plurality of assetsof the second feature to correspond with the modification to the firstasset in the first set of the plurality of assets corresponding to thefirst feature in accordance with the corrective shape; and displayingthe modified first asset in the first set of the plurality of assetscorresponding to the first feature and the updated second asset in thein the second set of the plurality of assets of the second feature. 6.The method according to claim 1, further comprising: retrieving thefirst set of the plurality of blend shapes from a persistent memory;storing the retrieved first set of the plurality of blend shapes to aworking memory; simulating a first expression of the avatar based on thefirst set of the plurality of blend shapes; retrieving the second set ofthe plurality of blend shapes from the persistent memory, wherein onlyblend shapes of the second set of the plurality blend shapes that arenot already in the working memory are retrieved from persistent memory;storing the retrieved second set of blend shapes to working memory; andsimulating a second expression of the avatar based on the second set ofblend shapes.
 7. The method according to claim 6, wherein one or moreblend shapes in the first set of blend shapes that are not shared withone or more blend shapes in the second set of blend shapes are movedfrom the working memory to the persistent memory.
 8. The methodaccording to claim 6, wherein the simulating the first expression of theavatar based on the first set of blend shapes comprises determining alinear combination of the first set of blend shapes for the first assetand a default shape for the first asset using an input position of theavatar.
 9. The method according to claim 1, wherein the second asset inthe first set of the plurality of assets of the first featurecorresponds with the first asset in the first set of the plurality ofassets corresponding to the first feature in response to the secondasset being active on the first feature of the avatar at a same time asthe first asset in the first set of the plurality of assets of the firstfeature.
 10. The method according to claim 5, wherein the correctiveshape is an offset value between 0 to 1.0.
 11. The method according toclaim 1, further comprising: generating a plurality of zones on theavatar, wherein the plurality of zones correspond to dynamic particleson the avatar; determining a first set of zones of the plurality ofzones corresponding with the first asset in the first set of theplurality of assets corresponding to the first feature; and identifyinga plurality of blend shapes for the first set of zones, wherein theplurality of blend shapes for the first set of zones corresponds to adirection of movement for each of the zones in the first set of zones.12. The method according to claim 11, wherein the dynamic particles onthe avatar correspond to predetermined areas of motion on the avatar.13. The method according to claim 11, further comprising: determining asecond set of zones of the plurality of zones corresponding with thesecond asset in the first set of the plurality of assets correspondingto the first feature; and generating a plurality of blend shapes for thesecond set of zones, wherein the plurality of blend shapes for thesecond set of zones include a plurality of degrees of movement for eachof the zones in the second set of zones, wherein one or more blendshapes of the plurality of blend shapes for the first set of zonesoverlaps with one or more blend shapes of the plurality of blend shapesfor the second set of zones.
 14. The method according to claim 11,wherein the direction of movement corresponds to one from a groupconsisting of an up movement, a down movement, a left movement, a rightmovement, a forward movement, and a backward movement.
 15. The methodaccording to claim 4, wherein the first feature is hair and the firstasset in the first set of the plurality of assets is a hairstyle. 16.The method according to claim 1, further comprising: in response toinitiating an application for generating the avatar, determining aplurality of features and a plurality of assets corresponding to theplurality of features most frequently used by a user; pre-computing theplurality of assets corresponding to the plurality of features mostfrequently used by the user; and caching the pre-computed plurality ofassets corresponding to the plurality of features.
 17. The methodaccording to claim 16, wherein the plurality of features most frequentlyused by the user is based on a history of user interaction with theapplication.
 18. The method according to claim 16, wherein in responseto the user selecting an asset from the pre-computed plurality of assetscorresponding to the plurality of features, completing rendering of theselected asset.
 19. A mobile device comprising: one or more processorsand a memory, wherein the one or more processors are configured to:store a plurality of assets and a plurality of blend shapescorresponding with the plurality of assets for a plurality of featuresof an avatar, wherein the plurality of assets are defined for a samerig; receive a selection of a first feature from among the plurality offeatures; identify a first set of the plurality of assets for the samerig that corresponds to the first feature; receive a selection of afirst asset among the first set of the plurality of assets correspondingto the first feature; identify a first set of the plurality of blendshapes corresponding to the first asset of the first feature that isselected among the first set of the plurality of assets corresponding tothe first feature; receive a selection of a second asset of the firstfeature among the first set of the plurality of assets corresponding tothe first feature; and identify a second set of the plurality of blendshapes corresponding to the second asset of the first feature that isselected among the first set of the plurality of assets corresponding tothe first feature, wherein one or more blend shapes of the first set ofthe plurality of blend shapes for the first asset of the first featureoverlaps with one or more blend shapes of the second set of theplurality of blend shapes for the second asset of the first feature. 20.The mobile device according to claim 19 wherein the storing theplurality of assets and the plurality of blend shapes corresponding withthe plurality of assets for the plurality of features comprises one ormore processors configured to: identify the first asset among the firstset of the plurality of assets corresponding to the first feature;obtain 3D geometry for the first asset among the first set of theplurality of assets corresponding to the first feature; compress the 3Dgeometry for the first asset; identify the first set of the plurality ofblend shapes corresponding to the first asset that is selected among thefirst set of the plurality of assets corresponding to the first feature;and store offset values for the first set of the plurality of blendshapes corresponding to the first asset.
 21. A non-transitory computerreadable medium including instructions configured to cause one or moreprocessors to perform operations comprising: storing a plurality ofassets and a plurality of blend shapes corresponding with the pluralityof assets for a plurality of features of an avatar, wherein theplurality of assets are defined for a same rig; receiving a selection ofa first feature from among the plurality of features; identifying afirst set of the plurality of assets for the same rig that correspondsto the first feature; receiving a selection of a first asset among thefirst set of the plurality of assets corresponding to the first feature;identifying a first set of the plurality of blend shapes correspondingto the first asset of the first feature that is selected among the firstset of the plurality of assets corresponding to the first feature;receiving a selection of a second asset among the first set of theplurality of assets corresponding to the first feature; and identifyinga second set of the plurality of blend shapes corresponding to thesecond asset of the first feature that is selected among the first setof the plurality of assets corresponding to the first feature, whereinone or more blend shapes of the first set of the plurality of blendshapes for the first asset of the first feature overlaps with one ormore blend shapes of the second set of the plurality of blend shapes forthe second asset of the first feature.
 22. The method according to claim1, wherein the first set of the plurality of blend shapes correspondingto the first asset of the first feature and the second set of theplurality of blend shapes corresponding to the second asset of the firstfeature are generated from a default mesh.